Skip to content

Keep the task instance in dag callbacks when its Dag version is missing - #72721

Merged
ephraimbuddy merged 3 commits into
apache:mainfrom
astronomer:callback-ti-dag-version-fallback
Sep 21, 2026
Merged

ephraimbuddy merged 3 commits into
apache:mainfrom
astronomer:callback-ti-dag-version-fallback

Conversation

@ephraimbuddy

Copy link
Copy Markdown
Contributor

Since 3.3.0 the scheduler dropped a task instance with no dag_version_id from the callback context rather than crash on it, which cost users who emit tracing events from on_success_callback / on_failure_callback the ti they correlate each event on, leaving them no way to tie a callback back to its task.

Handle a missing dag_version less destructively, so the callbacks receive the full task instance.


Was generative AI tooling used to co-author this PR?
  • Yes (please specify the tool below)

Generated-by: Claude Opus 5 following the guidelines


  • Read the Pull Request Guidelines for more information. Note: commit author/co-author name and email in commits become permanently public when merged.
  • For fundamental code changes, an Airflow Improvement Proposal (AIP) is needed.
  • When adding dependency, check compliance with the ASF 3rd Party License Policy.
  • For significant user-facing changes create newsfragment: {pr_number}.significant.rst, in airflow-core/newsfragments. You can add this file in a follow-up commit after the PR is created so you know the PR number.

@ephraimbuddy
ephraimbuddy force-pushed the callback-ti-dag-version-fallback branch 2 times, most recently from a7393ac to 8c2230e Compare September 8, 2026 16:05
Comment thread airflow-core/src/airflow/models/dagrun.py Outdated
Comment thread airflow-core/tests/unit/models/test_dagrun.py Outdated
Comment thread airflow-core/tests/unit/models/test_dagrun.py
@ephraimbuddy
ephraimbuddy force-pushed the callback-ti-dag-version-fallback branch from 8c2230e to a6c2887 Compare September 10, 2026 08:59
Comment thread airflow-core/src/airflow/models/dagrun.py
Comment thread airflow-core/src/airflow/models/dagrun.py Outdated
Comment thread airflow-core/tests/unit/models/test_dagrun.py Outdated
Comment thread airflow-core/tests/unit/models/test_dagrun.py Outdated
@ephraimbuddy
ephraimbuddy force-pushed the callback-ti-dag-version-fallback branch 2 times, most recently from 5c73b5e to c91d07f Compare September 14, 2026 16:18

@ferruzzi ferruzzi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks like everything has been addressed, thanks.

Since 3.3.0 the scheduler dropped a task instance with no dag_version_id from the
callback context rather than crash on it, which cost users who emit tracing events
from on_success_callback / on_failure_callback the ti they correlate each event on,
leaving them no way to tie a callback back to its task.

Handle a missing dag_version less destructively, so the callbacks receive the full
task instance.
settings.Session is a scoped_session, so a defaulted session hands back
the caller's own and then commits and closes it out from under them.
Every production caller already holds a session, so the default bought
nothing and only stood to end a transaction nobody asked to end.
One warning covered both stand-in tiers, so an ordinary Airflow 3 task
instance whose run still has a version was reported to operators as a
record predating Dag versioning. Both tiers also warned even though the
callback receives its full context and nothing is lost; the sibling
helper in the scheduler reserves warning for the case it cannot resolve.

The tests covering the fallback let get_task_instance default its
session, which committed and closed the fixture's own. The objects came
back detached, so the flush never reached the database and no versionless
row was ever under test.
@ephraimbuddy
ephraimbuddy force-pushed the callback-ti-dag-version-fallback branch from c91d07f to 9f189de Compare September 21, 2026 06:38
@ephraimbuddy
ephraimbuddy merged commit 01fc0c5 into apache:main Sep 21, 2026
78 checks passed
@ephraimbuddy
ephraimbuddy deleted the callback-ti-dag-version-fallback branch September 21, 2026 07:56
@ephraimbuddy ephraimbuddy added this to the 3.3.3 milestone Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants